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TWO-STEP COMMERCIAL RECOMMENDATION 



The invention relates generally to commercials in audio and/or video signals such as 
television or radio signals and, more particularly, to a method and apparatus for 
personalizing the commercials in such signals for a user. 
5 In order to improve the user's experience, and to make commercials more effective, one 
can replace the commercials in a live broadcast stream with personalized commercials. 
U.S. patent 6,177,960 to Van Luyt for a "TV signal receiver", issued Jan. 13, 2001, and 
incorporated herein by reference, discloses one possible scheme for replacing the 
commercials in a live broadcast stream with personalized commercials. For example, the 

10 personalized commercial can be selected based on the demographic factors of the target 
audience of a particular TV or radio program. Such factors may include, e.g., gender^ 
age, income and geographic location, and may be predicted based on the content of the 
program using known research and survey techniques. Advertisers can then choose to 
advertise their products on particular programs whose demographic factors correlate with 

15 those of the product of service being offered. This correlating, however, is much too 

rough to provide truly personalized results since the interests of each individual can vary 
widely in ways that cannot be predicted by the demographic factors. 
Accordingly, it would be desirable to provide a method and apparatus for personalizing 
the commercials that are displayed to a user in commercial breaks of a video and/or audio 

20 program. 

In a particular aspect of the invention, a method is provided for selecting personalized 
commercials. The method includes providing, for each of a plurality of programs, a score 
indicating a degree of preference of at least one user in relation thereto; providing, for 
each of a plurality of commercials, respective correlation factors indicating respective 
25 degrees of effectiveness in relation to each of the plurality of programs; and providing, 
for each of the plurality of commercials, a metric indicating a degree of effectiveness in 
relation to the at least one user based on the scores and the respective correlation factors. 
The at least one user may be, e.g., an individual person, or a group of people in a 
household. 

30 A related apparatus and program storage device are also provided. 
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In the drawings: 

Fig. 1 illustrates an embodinfient of an apparatus for recommending commercials; 
Fig. 2 illustrates linking a user to programs and commercials; and 
Fig. 3 illustrates an embodiment of a method for recommending coimnercials. 
5 In all the Figures, corresponding parts are referenced by the same reference 

numerals. 

The present invention involves making commercial recommendations much more 
personalized. To this end, a two-step linking between a user and a set of commercials is 
provided. The first step is to determine the relation between the user and a set of 

10 programs, such as television or radio programs, or other video, audio or audio/video 
programs. This can be achieved, e.g., using a program recommender. This relation 
indicates how much the user likes each of the programs. The second step is to determine 
the relation between the set of programs and the set of commercials. This can be 
achieved, e.g., based on the advertiser's knowledge. Then, Ihe link between the user 

15 (e.g., viewer/listener) and each of the commercials can be determined based on the 

relations between the user and the programs, and the relations between the programs and • 
tiie commercials, to identify one or more personalized commercials for the user. 

Fig. 1 illustrates an embodiment of an apparatus for reconunending commercials. 
In one possible approach, the invention is implemented using components within a • 

20 television set-top box receiver that receives a television signal and outputs a signal for 
display on a television. However, the invention is generally applicable to any type of 
device that receives video programs, audio programs, or audio/video programs. For 
example, the invention may be implemented in a computer that receives audio/video 
programs from a networic such as the Internet, e.g., by downloading, streaming or 

25 broadcast. The video programs typically include an audio track although this is not 
required. Audio-only programs may include the audio track of a radio program, for 
example. Generally, the programs may be provided by any source, including the Internet, 
cable, and terrestrial or satellite broadcasts. The programs may include pay-per-view 
programs. The programs may be played as they are received, or stored for subsequent 

30 playing. 
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The present example refers to a video program for illustration only. In one 
approach, the receiver 100 demultiplexes and decodes the received video programs at a 
demultiplexer/decoder 110. The video programs may be provided in a digital or analog 
multiplex that is transmitted by cable, satellite, or terrestrial broadcast, for example. 
5 Generally, one of the video programs is decoded based on a channel selection made by 
the user/viewer via a user interface 130, The decoded video program may be 
conmiunicated to a display device 190 via a CPU 140, which includes a working memoiy 
150, or stored locally for subsequent display, e.g., at a video storage device 1 15. In one 
possible design, the working memoiy ISO is a program storage device that stores 

10 software that is executed by the CPU 140 to achieve the functionality described herein. 
However, resources for storing and processing mstructions such as software to achieve 
the desired functionality may be provided using any known techniques. 

A commercial stomge device 120 stores a number of commercials, which may be 
received at the receiver 100 using any known technique. For example, the commercials 

15 may be received and stored over time with the broadcast of the video programs. The 

commercials may be received via the same or a separate communication path from which 
the programs are received. Particular ones of the commercials are selected for insertion 
into conmiercial breaks in the video programs using the techniques disclosed herein. The 
personalized commercial insertion can use the local commercial storage 120 to play out a 

20 commercial. Multiple commercials in a broadcast signal may be another way to offer a 
choice of commercials to the viewers. A dedicated channel may carry commercials from 
which selections can be made. 

Various techniques for inserting personalized commercials into video programs 
arc described in, e.g., WO 98/36563 to Van Luyt, published August 20, 1998, entitled 

25 *TV signal receiver*'; WO 01/08406 to Lambert et al., published February 1, 2001, 

entitled 'TV signal receiver*'; and U.S. 2002/0131772 to Vrielink, published September 
19, 2002, entitled "Methods of and devices for transmitting and reproducing audio and/or 
video information consisting of primary programs and commercials", and the 
aforementioned U.S. patent 6,177,960 to Van Luyt for a 'TV signal receiver", each of 

30 which is incorporated herein by reference. Many of these techniques apply to programs 
having video and/or audio portions. 
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A program recommender 160 provides information that indicates the degree of 
preference by a particular user, or a group of users, for different programs or shows - that 
is, the extent to which a user enjoys, or is expected to enjoy, watching a program. The 
preference can be provided for a currently watched program, or programs that are 
5 scheduled for future viewing, using any recommender technique. For example, one may 
use an explicit recommender, where a user indicates explicitly what aspects of programs 
the user likes or dislikes. The user may indicate that he or she likes programs related to 
comedy, or specific sports events, or programs with specific actors, and so forth. Or, one 
may use an implicit recommender that learns the user's likes and dislikes from the user's 

10 viewing/listening histoiy. For example, the user can indicate via an interface that he likes 
or dislikes particular programs, and the recommender can extrapolate that information to 
determine whether the user would like or dislike other programs. The program 
recommender 160 may also predict the user's degree of preference for a program based 
on demographic information such as the user's gender, age, location, income and the like. 

15 For more information on program reconunenders, see, e.g., U.S. patent 

application no. 09/466,406 to Srinivas Gutta, entitled "Method and Apparatus for 
Reconunending Television Progranmiing using Decision Trees," filed December 17, 
1999 (Disclosure No. 700772), and U.S. patent application no. 09/666,401 to Kaushal 
Kurapati, Dave Schaffer and Srinivas Gutta, entitled 'Method and Apparatus for 

20 Generating Recommendation scores using Implicit and Explicit Viewing Preferences", 
filed September 20, 2000 (Filing No. US000239, Disclosure No. 701247), both of which 
are incorporated herein by reference. Many of these techniques apply to programs having 
video and/or audio portions. 

The commercial classifier 170 provides information to the CPU 140 regarding the degree 
25 of effectiveness of a conmiercial in relation to a particular program. Generally, this 
information is available from advertisers, and reflects the degree of success of running 
the commercial in the particular program, e.g., based on resulting sales or sales inquiries, 
surveys, or other metrics. This information involves the effectiveness of flie commercial 
in the particular program for all users. The present invention advantageously enables the 
30 effectiveness of a commercial to be determined for a particular user or a small group of 
users such as a family. 
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Based on the infonnation from the program recommender 160 and the commercial 
classifier 170, the CPU 140 calculates an overall metric tiiat is used to identify one or 
more conomercials to display to the user in commercial breaks of a program currently 
being played on a display or other output device, or played at a future time. The 
5 commercials may be identified by a codeword identifier associated with each commercial 
or usmg any other scheme that is known in the art. When the commercials are stored in 
the commercial storage 120, they may be identified and located using any known 
memoiy management or database storage techniques. See, for example, the 
aforementioned WO 98/36563, WO 01/08406, U.S. 2002/0131772, and U.S. patent 
10 6,177,960. 

Note that the configuration shown in Fig. 1 is a simplified example. Moreover, the 
various components that store and process information need not be distinct components 
but their functions can be combined and carried out by common processing and storage 
elements. 

15 Fig. 2 illustrates linking at least one user 205 to a number of programs 210, 212, 214, 
216, 218, ... and to a number of commercials 260, 262, 264, 266, ... The program 
recommender 160 may provide the mformation that indicates the degree of preference of 
the user 205 for the programs 210, 212, 214, 216, 218, ... as numeric weights or scores 
denoted as w(user, show_t), where 'Sv" denotes '^weight", "user'' denotes a particular 

20 user, and showjt denotes a particular rth show, where t is an index representing each 

program. For example, the programs 210, 212, 214, 216, 218, ... may be denoted by t?=l, 
2, 3, 4, 5, ... The weights may range between zero and one, for instance, indicating low 
and high preferences of the user 205 for a program. As an example, the user may have a 
preference of 0.9 for program 210, and a preference of 0,6 for program 212. Not all 

25 weights are shown. The preferences may be obtained from an explicit and/or implicit 
recommender or other techniques, as discussed previously. 

The programs may be any one-time or series, e.g., recurrent, programs. For instance, 
program 210 may be the weekly news magazine "60 minutes", program 212 may be the 
weekly situation comedy program "Everybody loves Raymond", program 214 may be a 
30 bi-weekly "movie special," program 216 may be a daily re-run of *The Simpsons," and 
program 218 may be the daily program "The Tonight Show". 
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Similarly, the commercial classifier 170 may provide the information that indicates the 
effectiveness of a commercial relative to a program as numeric weights or correlation 
factors denoted as w(showjt, comm_i), where "commj" denotes a particular /th 
commercial, where i is an index representing each commercial. For example, the 
5 commercials 260, 262, 264, 266, ... may be denoted by i=l, 2, 3, 4, ... , respectively. 
Moreover, t is an index representing each program. For example, the programs 210, 212, 
214, 216 and 218 ... may be denoted by t?=l, 2, 3, 4, 5 respectively. The correlation 
factors may range between zero and one, for instance, indicating low and high 
correlations, respectively, of the commercial relative to a program. As an example, the 

10 commercial 260 may have correlation factors of 0.3, 0.7, 0.1, 0.2 and 0.05 relative to 
programs 210, 212, 214, 216, 218, respectively. Not all correlation factors are shown. 
Generally, Ihe advertiser associated with a commercial can determine, e.g., from 
consumer surveys and other research, how strongly a cottmiercial is linked to the target 
audience of each program, and obtain a corresponding correlation fector. The correlation 

IS factor of a commercial may indicate a return on investment based on the dollar amount of 
sales generated from the conimercial versus the amount of advertising dollars spent for 
tihe commercial time on a given program. Thus, the correlation factor for a commercial- 
program combination can be set by the advertiser, in one possible approach. This 
information can be communicated to the receiver 100 with a TV broadcast, for example, . 

20 or using other techniques, such as by download via the Internet. To provide a specific 
illustration, assume the commercial 260 is for a particular coffee brand, commercial 262 
is for a particular automobile, commercial 264 is for a particular beverage, and 
commercial 266 is for a particular line of clothing. 

In accordance with the invention, the user-program weights or scores and the program- 
25 commercial correlation factors, are used to obtain a metric that correlates an effectiveness 
of the commercials to the individual user. This is achieved for each commercial by 
summing the product of the weights and correlation factors over each program. For 
example, for the commercial 260, the metric is calculated as (0.9 x 0.3) + (0.6 x 0.7) + 
(0.3 X 0.1) + (0.5 X 0.2) + (0.7 x 0.05) == 0.855. Generally, for each commercial, the 
30 metric that correlates an effectiveness of the commercials to the individual user/viewer 
can be calculated as: 
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n- p rog rams 

For all i: link(user, comm_i)= ^ w(user, show_t) • w(show_j;, comm_i) 

Where n-programs is the number of programs. Stated alternatively, for each of 
the commercials, the degree of effectiveness (E) in relation to the viewer can be provided 

n-programs 

as: E= scoreit) • correlation factor{t) , where t is an index denoting each rth video 

5 program, where t=l, n-programs, score(t) denotes the score of the /th video program, 
and correlation factor(t) denotes the correlation factor relative to the rth video program. 
As one can see from the formula, a commercial gets a high degree of effectiveness if it 
has a high correlation factor to many programs that the user likes. For instance, sports 
commercials for yoimgsters can be correlated by the advertiser to both sports programs 
10 and programs for young people, and by the above formula flie commercial will indeed get 
a high computed effectiveness for people that like both kind of programs. The 
commercials with the highest metric values can be recommended for future display to the 
user. 

Fig. 3 illustrates an embodiment of a method for recommending commercials. 

15 The process starts at block 300 with the first commercial, and at block 305 with the first 
program. At block 310, the effectiveness metric (E) is initialized to zero. At block 320, a 
correlation factor (CF) is obtained indicating an effectiveness of the first commercial 
relative to the first program, ranging, e.g., from zero for "least effective" to one for "most 
effective". At block 330, a score is obtained indicating the user's degree of preference 

20 for the current program, e.g., 0 for "hates", 0.1 for "strongly dislikes", 0.2 for 

"moderately dislikes", 0.5 for **neutral", 0.8 for "moderately likes", 0.9 for 
"strongly likes", and 1.0 for "loves" At block 340, the effectiveness metric (E) is 
calculated from E=E + (CF»S). 

At block 350, it is determined if there are programs remaining that have not been 

25 processed. If so, the correlation factor relative to the next program is obtained at block 
320, the score indicating the user's degree of preference is obtained at block 330, and the 
effectiveness metric is updated at block 340. Once the last program has been processed, 
as determined at block 350, the effectiveness metric (E) for the current commercial is 
stored. This is the final metric value for the current commercial. 
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If there are additional commercials to process, as determined at block 370, the 
next commercial is processed starting at block 305 with the first program. At block 310, 
the effectiveness metric (B) is reset to zero. The process continues as discussed above 
until all commercials have been processed to obtain an effectiveness metric. At this time, 
5 one or more commercials are recommended for display to the viewer at block 380, e.g., 
based on the commercials with the highest effectiveness metrics. 
Note that the programs can be scored without determining which program is currently 
being played. A preference score can be predicted for each (future) show and a 
conunercial can be reconunended based on the effectiveness (E) metric, e.g., so that the 

10 commercials with the highest effectiveness metrics are reconunended. Similarly, there is 
no need to monitor the commercials that are currently being played. Note that an 
identifier should be provided for each commercial for which a correlation factor is 
obtained so particular ones of the commercials that are recommended for display to a 
viewer can be easily identified. 

15 When the personalized commercials are displayed, they can be correlated to the programs 
in which they were previously run so that they run again in the same program. The 
program in which the personalized commercials run may be a subsequent presentation of 
a recurrent weekly program or in a similar type of program, e.g., programs in the 
category of situation comedies or sports events. Or, the personalized commercials need 

20 not be correlated to the programs in which they were previously run, in which case they 
can be shown when the viewer views any subsequent program. Factors such as time of 
day or day of week can also be considered so that, e.g., a commercial that has run at a 
particular time of day and that is found to be particularly effective relative to a user can 
be run again at the same time of day in a subsequent day. For example, a coffee 

25 commercial may be more effective in the moming when more people drink coffee. 
Note also that the process of Fig. 3 can be completed for each user or for a group of 
users. For example, multiple users in a home may each be identified by an id number or 
other identifier that they provide when viewing programs via the user interface 130. In 
this way, the commercials can be personalized for the current user or a group of users 

30 (e.g., a family). For a group of user, step 330 may be modified to use an average of the 
preferences of the different users, such as a uniform average or a weighted average. 
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Moreover, the process of Fig. 3 may be repeated from time to time to reflect changes in 
the scheduled programs and/or in the commercials. 

While there has been shown and described what are considered to be preferred 
embodiments of the invention, it will, of course, be understood that various modifications 
5 and changes in form or detail could readily be made without departing from the spirit of 
the invention. It is therefore intended that the invention not be limited to the exact forms 
described and illustrated, but should be construed to cover all modifications that may fall 
within the scope of the appended clauns. 



